Integrating Motion Tracking Sensors to Human-Computer Interaction with Respect to Specific User Needs

This paper presents a novel framework for combining Microsoft Kinect device (Kinect) and Leap Motion sensor with Java Monkey Engine and utilizing it for natural human computer interaction. The framework supports the standard input devices (keyboard, mouse) as well as selected motion tracking sensors Microsoft Kinect, Leap Motion). To demonstrate its aplicability, the framework was subsequently used for creating the demonstration application called Labyrinth. This application was designed according to the current requirements of psychologists from the Department of Psychology at the Faculty of Arts, Masaryk University, with respect to children with specific disorders, such as ADHD (Attention Deficit Hyperactivity Disorder). Similar kinds of applications are widely utilized by psychologists for observing and evaluating their impact on various groups of patients. To make the application more realistic, our framework integrates the stereo projection and the jBullet physics library for simulation of the gravitational force applied to the scene. The resulting application forms a comprehensible basis for future applications based on this framework.